5 CONVOLUTION OF MEASURES ON A LOCALLY COMPACT GROUP 271
(c) Let T? be the set of measures a e T0 such that G/Ha is compact. Show
that T°(g) <= r? for all ge Jf+(G). Let ae r?, and let ge Jf+(G) be such that
\g(xs) doc(s) = 2 for all x e G (Problem 2(a)). For each compact L <= G, the set W
of measures ]8 e r° such that #(xs) d)8($) ^ 1 for all x e L is a neighborhood of a
in F° with respect to the vague topology. Show that if G is generated by a compact
neighborhood U of e, and if we take L = UK above, then W <= r°(#). Deduce that
the restriction to F° of the mapping <xh-> ||/z«|| is vaguely continuous in this case.
(d) Let rd c F° be the set of measures a such that Ha is discrete, and let N be the
subset of Fd consisting of measures a such that a({<?}) = 1. For each relatively compact
open neighborhood U of e in G, let NU be the set of all a e N such that Ha n U = {e}.
Show that NU is compact (observe that the relation a e NU is equivalent to a({e}) ^ 1
and a(U) < 1). As U runs through the set of relatively compact open neighborhoods
of e in G, the interiors of the sets NU cover N. A subset M of N is relatively compact
in N if and only if there exists a relatively compact open neighborhood U of e in G
such that M (
7. With the notation of Problems 5 and 6, suppose that there exists a neighborhood of e
in G which contains no finite subgroup of G other than {e}. Show that the mapping
at-* a({e}) of Td into R* is vaguely continuous. (Show that there exists a neighborhood
V of e in G and a neighborhood W of a in Td such that the relation j8 e W implies
(V2 — V) n H0 = 0: argue by contradiction.)
8. With the notation of Problems 5 and 6, suppose that G is commutative and generated
by a compact neighborhood of e. Let Nc denote the subset of N consisting of measures
a such that Qa = G/Ha is compact, so that Nc = N n F?; the set Nc is open in N
(Problem 6(c)). A subset A of Nc is relatively compact in Nc if and only if it satisfies
the following two conditions: (1) there exists an open neighborhood U of e in G such
that Ha n U = {e} for all a e A; (2) there exists a constant k such that jua(G/Ha) ^ k
for all a e A. (Use Problem 6 and the fact that p,x is a Haar measure on G/H«.)
9. Translate the results of Problems 6 to 8 into statements about subspaces of the space
S of closed subgroups of G (Problem 5(c)): in particular, the subspace S° of unimodular
closed subgroups, the subspace X? of subgroups H e S° such that G/H is compact,
the subspace D of discrete subgroups, and the subspace Dc = D n S?. In particular,
obtain Mahler's criterion: if for each discrete subgroup H e Dc we denote by u(H)
the total mass of G/H relative to the measure /xa corresponding to the Haar measure
a on H such that a({e}) = 1, then a subset A of Dc is relatively compact in Dc if and
only if (1) there exists a neighborhood U of e in G such that H n U = {e} for all
H e A; and (2) there exists a constant k such that u(H) 5* k for all H e A. Consider
the case G = R".
5. CONVOLUTION OF MEASURES ON A LOCALLY COMPACT GROUP
Let fj,i9..., nn be a finite number of (complex) measures on a locally com-
pact group G. The sequence (jj,l9..., /O is said to be convolvable if, for each
function/e ^C(G), the function